A virtual reality (VR) system replicates an environment that simulates physical presence in places in the real world or an imagined world using entirely-computer-generated three-dimensional (3D) imagery of “scenes” within this world. Similarly, an augmented reality (AR) system “augments” physical presence in the real world through computer-generated 3D imagery that overlies contemporaneously captured imagery of the real world. Thus, VR and AR systems both seek to provide an accurate sense of “presence” in the real, augmented, or imagined world. Typically, this sense of presence is facilitated through the use of a head mounted display (HMD) device that provides separate left-eye and right-eye displays. The displays together present a stereoscopic, or 3D, representation of a scene in the represented world, where the presented scene reflects the user's relative perspective of the scene based on the user's current pose (that is, the location and orientation of the user's head relative to a reference coordinate frame for the depicted scene).
HMD-based VR and AR systems display 3D imagery as a sequence of display textures (or “frames”), each display texture rendered based on a corresponding detected head pose and persisting for a particular period of time. However, as an HMD device generally permits a user to move freely about, the user's head may have perceptibly moved in the time between the initiation of the rendering of a texture and the display of the resulting rendered texture. As such, the imagery displayed at the HMD device at a particular point in time may lag behind the user's head movements. Further, virtual objects in the scene may be “in motion” and thus the rendered positions of the virtual objects may differ from where the objects should appear if the frame was rendered at the correct time. This dissonance between the user's perceived orientation within a scene, as well as perceived orientation of objects within that scene, and the actual orientation of the scene and objects contained therein can lead to user disorientation or discomfort, or what is often referred to as “virtual reality sickness”. Thus, to reduce or eliminate user discomfort and thereby provide improved presence, HMD-based VR and AR systems seek to minimize the motion-to-photon latency; that is, the latency between a user head/eye movement and when photons representing a scene from the resulting new pose hit the user's eye.